Improving Tree-Structured Decoder Training for Code Generation via Mutual Learning

Code generation aims to automatically generate a piece of code given an input natural language utterance. Currently, among dominant models, it is treated as sequence-to-tree task, where decoder outputs sequence actions corresponding the pre-order traversal Abstract Syntax Tree. However, such only exploits based preceding actions, which are insufficient ensure correct action predictions. In this paper, we first throughly analyze context modeling difference between neural models with different traversals decodings (preorder vs breadth-first traversal), and then propose introduce mutual learning framework jointly train these models. Under framework, continuously enhance both two via distillation, involves synchronous executions one-to-one knowledge transfers at each training step. More specifically, alternately choose one model student other its teacher, require fit data prediction distributions teacher. By doing so, can fully absorb from thus could be improved simultaneously. Experimental results in-depth analysis on several benchmark datasets demonstrate effectiveness our approach. We release https://github.com/DeepLearnXMU/CGML.